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Abstract: This study explored the 

performance of 75 seventh-grade students 
with and without disabilities, educated in 
inclusive mathematics classes, on open-ended, 
problem-solving mathematics assessments. In 
the study, approximately half of the students 
used a graphing calculator on the first 
assessment and not on the second assessment 
[n — 35; 46.7%), whereas the other half used it 
on the second assessment and not on the first 
(n = 40; 53.3%). The results indicate that all 
students did better when using a graphing 
calculator, regardless of the order of calculator 
use (i.e.. Assessment 1 or 2). The results also 
suggest that calculators may not be a valid 
accommodation for some students with 
disabilities on assessments. This study has 
implications for providing calculators as 
accommodations on mathematics 

assessments. 

Keywords: Mathematics, Calculators, High 
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Accountability is at the forefront of education 
and so is its ‘sidekick’- assessment. Federal 
policy requires that all students be tested 
yearly in literacy and mathematics in grades 3 
through 8 and once again between grades 10 
and 12 (No Child Left Behind Act of 2001 
[NCLB]). Although all students are to be 
tested, students are not all the same. Students 
with disabilities, for example, often stmggle 
with content areas, such as mathematics, and 
therefore perform worse on assessments 
(Fuchs, Fuchs, & Capizzi, 2005). One way to 
better measure the performance of students 


with disabilities on assessments is to provide 
an accommodation or set of accommodations. 

Accommodations are a right of students with 
disabilities on assessments and in daily class 
activities (Individuals with Disabilities 
Education Improvement Act of 2004 
[IDEIA]; Koening & Bachman, 2004). A valid 
accommodation does not alter the construct 
of an assessment, but rather alters the 
presentation, type of response, setting, timing, 
or the provision of technology or other 
supports, according to a student’s individual 
needs (Fuchs et al., 2005; Ysseldyke, Thurlow, 
McGrew, & Shriner, 1994). Accommodations 
help ‘level the playing field’ for students with 
a disability (Fuchs & Fuchs, 1999). Examples 
of common accommodations for students 
with disabilities, particularly students with a 
high incidence disability, include: tests read 
aloud, allowing oral responses, calculators, 
individual administrations, and extended time 
(Thurlow, Elliott, & Ysseldyke, 2003). 

While accommodations for students with 
disabilities are varied and can be expansive, 
not every accommodation is a valid 
accommodation for assessments. Elbaum 
(2007) defined a valid accommodation as one 
that results in the performance of students 
with disabilities increasing to a greater extent 
than the performance of students without 
disabilities when provided with the same 
accommodation on the same assessment. 
Given the proliferation of assessing students 
in the era of accountability, research exploring 
the validity of accommodations has increased. 
Yet, even with this increased attention, there 
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is still a dearth of research. Additional studies 
are needed to understand the validity of 
different accommodations, specifically on 
mathematics assessments, and calculators are 
a natural option given their frequent 
appearance as accommodations on individual 
education programs (IEPs; Maccini & 
Gagnon, 2000; Thurlow, Lazarus, Thompson, 
& Morse, 2005; Tindal & Ketterlin-Geller, 
2004). 

Accommodations for Mathematics Assessments 

Researchers have studied the validity of 
particular accommodations relevant to 
mathematics assessments, such as extended 
time, oral presentation, and calculators. The 
research on extended time has shown to be 
mixed and dependent on the type of 
mathematics problems being assessed (Fuchs, 
Fuchs, Eaton, Flamlett, & Karns, 2000). 
Fuchs and colleagues found that extended 
time is not a valid accommodation on 
computation mathematics assessments or 
those involving application problems, as 
students with disabilities did not benefit on 
Curriculum-Based Measurement assessments 
more than students without disabilities when 
provided with this accommodation (i.e., did 
not improve scores more). Flowever, the 
researchers did find statistically significant 
benefits for extended time when students with 
and without disabilities took a problem- 
solving assessment, favoring students with 
disabilities. (Note: This suggests that extended 
time is a valid accommodation for problem- 
solving assessments.) 

Researchers also have found mixed results for 
the oral presentation of mathematics 
assessments as an accommodation for 
students with disabilities. Similar to the 
extended time accommodation, Fuchs et al. 
(2000) found no benefit in the area of 
mathematics for students with disabilities on 
application assessments, but did find 
statistically significant benefits for this 


accommodation on the problem-solving 
assessments. Tindal, Fleath, Flollenbeck, 
Almond, and Flarniss (1998) found that 
students with disabilities benefited when a 
mathematics assessment was read aloud by a 
teacher as opposed to students with 
disabilities themselves reading the test but this 
was not the case for students without 
disabilities, making oral presentation an 
appropriate accommodation. 

Finally, and similar to the other 
accommodations examined with respect to 
mathematics assessments, inconsistent results 
have been found for calculators as an 
accommodation. Fuchs et al. (2000) examined 
the use of a calculator as an accommodation 
for fourth- and fifth-grade students with 
learning disabilities on curriculum-based 
measurements (CBMs). Their research 
involved 181 students without disabilities and 
192 students with learning disabilities. 
Students were given computation, concepts 
and applications, and problem-solving CBMs. 
The researchers found that students with 
learning disabilities benefited more than their 
peers without disabilities when using a 
calculator on problem-solving CBMs. 
Flowever, the students with disabilities did not 
benefit more than students without disabilities 
on the concepts and application CBMs. 
(Note: Fuchs et al. study examined several 
different types of accommodations and the 
overarching study was to compare teacher- 
alone vs. data-support accommodation 
decisions.) 

Shaftel, Belton-Kocher, Glasnapp, and Poggio 
(2003) also studied the impact of calculators 
as an accommodation for students with and 
without disabilities. Specifically, they studied 
570 fourth graders with disabilities and 244 
sixth graders without disabilities. Shaftel and 
colleagues found the use of a calculator 
benefited students with disabilities but not 
students without disabilities and concluded 
that calculators were an appropriate 


Assistive Technology Outcomes and Benefits / 20 



Fall 2008, Vol.5, Num. 1 


accommodation for students with disabilities. 
However, the results were not conclusive as 
the assessments used for students with 
disabilities in the study were presented in 
simpler English in addition to students being 
provided a calculator. Furthermore, students 
with and without disabilities were not tested at 
the same grade levels. 

Recent research has explored calculators-both 
four-function and graphing calculators-as an 
accommodation on open-ended problem- 
solving assessments. Bouck and Bouck (2008) 
studied four-function calculators as a 
mathematics assessment accommodation. The 
research involved 89 sixth graders with and 
without disabilities on open-ended, problem- 
solving, number and operation, time-limited 
assessments. They found that the use of a 
standard four-function calculator resulted in 
both students with and without disabilities 
answering more questions correctly when they 
had access to the calculator on the assessment 
than when not. However, students with 
disabilities did not benefit more than students 
without disabilities when provided with this 
accommodation. 

Similar results were found by Bouck (in press) 
in examining graphing calculators as an 
assessment accommodation by students with 
and without disabilities. This study analyzed 
the performance of 47 seventh-grade students 
with and without disabilities, in inclusive 
mathematics classes, on an open-ended, 
problem-solving, number and operation, time- 
limited mathematics assessment. While the 
data showed that on the problem-solving 
assessments, students with disabilities 
answered more problems correctly when 
given access to a graphing calculator, these 
gains were not statistically significant when 
compared to students without disabilities. 

This specific research project sought to 
continue and extend the research on 
calculators as an accommodation on 


assessments. It focused on graphing 
calculators and the order in which students 
were allowed access to a calculator (first 
assessment or second assessment). In 
particular, it sought to answer the following 
question: Does the use of a graphing 

calculator result in performance differences 
on standards-based, open-ended, problem- 
solving assessments for students with and 
without disabilities? 

Method 

Participants 

Seventy-five seventh-grade students 
participated in this study. All participants 
came from two schools in one large mral 
district in a midwestern state. The district was 
selected because it had been using a problem- 
centered mathematics curriculum which 
encouraged calculator use for over a decade. 
It also educated the majority of its students 
with a disability in inclusive mathematics 
classes. The two schools had a combined 
student population of 2,577 students, an 
average rate of 93.1% Caucasian students, an 
average rate of 78% passing the state 
mathematics assessment, and an average rate 
of 88.3% passing the state reading assessment 
(School Matters, 2006). The district as a whole 
had a 28% economically disadvantaged rate 
and 13.8% of its students identified with 
disabilities (School Matters). 

Four inclusive classes and two teachers (both 
general education mathematics teachers) 
participated in this study. Fifty-three percent 
of the students had Teacher A [n — 40) and 
47% Teacher B {n — 35). The students were 
relatively evenly dispersed across the four 
classes. Of the 75 students who completed 
both assessments, 74.7% ( n — 56) were 
students without disabilities and 25.3% (n = 
19) were students with high incidence 
disabilities. The majority of the students with 
disabilities were diagnosed with a learning 
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disability (» = 13; 68.4%) (NOTE: The 
schools did not indicate the type of learning 
disability students had, such as a learning 

disability related to reading, writing, 

mathematics, and other subject areas); 
however, others included students with 
Attention Deficit Hyperactivity Disorder 

(ADHD; n — 5; 26.3%) and students with 
behavior disorders / emotional impairments 
(n — 2; 10.5%). Slightly more than half of all 
the students were female [n — 40; 53.3%), yet 
only 31.6% {n — 6) of the students with 
disabilities were female. 

Materials 

All students in the study completed the same 
two assessments in the same order 

(instmments available upon request from the 
author) as well as used the same type of 
calculator (a TI-82 graphing calculator), which 
was the standard calculator for these students 
and all students were familiar with it and had 
used it previously. The two assessments were 
similar but not identical. Both assessments 
consisted of 28 open-response, problem- 
solving questions that focused on the number 
and operation strand from the National 
Council of Teachers of Mathematics (2000) 
Principles and Standards for School Mathematics. 
The number and operation strand was chosen 
for both assessments because the majority of 
the standards for sixth grade students in this 
midwestern state came from this strand. The 
state’s sixth-grade standards were chosen as 
students were tested at the beginning of their 
seventh-grade year and testing students on the 
sixth grade standards would reflect what 
students were suppose to have learned 
following the completion of their previous 
year of schooling. The assessment questions 
represented adaptations of released items 
from the state’s general large-scale assessment 
(Michigan Department of Education, 2006) 
and released items from the National 
Assessment of Educational Progress 
(n.d.).The assessments were reviewed by well- 


known mathematics education specialists in 
the state for clarity, appropriateness, and 
alignment to state standards. 

Procedure 

The study involved two assessments taken 
about four weeks apart. Both assessments 
were timed, in that students had one class 
period to complete the assessment (50 
minutes across all classes). About half of the 
students ( n — 35; 46.7%) were assigned to 
Condition 1, meaning that they had access to 
a graphing calculator on the first assessment 
(Assessment 1) but not the second. The other 
students ( n — 40; 53.3%) were assigned to 
Condition 2, in which they had access to a 
graphing calculator on the second assessment 
(Assessment 2) and not on the first. The 
students were randomly assigned to a 
condition (i.e., order of calculator use) at the 
level of teacher, which means that students 
themselves were not randomly assigned to use 
a calculator or not, but a class was assigned to 
use a calculator or not on the assessment (see 
Figure 1 for graphical depiction of 
conditions). 

Data Analysis 

The mathematics assessment data were 
analyzed multiple ways. First, the data was 
analyzed using a 2 x 2 AN OVA (Ability 
status x Condition). The dependent variable 
was students’ raw change score from 
Assessment 1 to Assessment 2 and was 
computed by subtracting the number of 
questions students answered correctly on the 
first assessment (out of 28) from the number 
students answered correctly on the second 
assessment (out of 28). The change score was 
selected as the dependent variable following 
Richards’s (1975) argument that change scores 
representing the difference between pretest 
and posttest are appropriate, easier to 
compute, and have greater meaning to non- 
researchers. Ability status (students with 
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Figure 7: Graphical representation of study conditions. 




disabilities and students without disabilities) 
and condition (calculator use on Assessment 1 
or calculator use on Assessment 2) were the 
two factors in the AN OVA. 

Independent t-tests were also completed for 
each condition with ability status as a factor. 
For Condition 1 (access to a calculator on 
Assessment 1), the dependent measure was 
students’ scores on the first assessment when 
the graphing calculator was used. For 
Condition 2, the dependent measure was 
scores on the second assessment when 
students had access to a graphing calculator. 
The mathematics assessments data also were 
analyzed using frequency distributions. 

Results 

Analyzing students’ change scores on the 
mathematics assessments from the first 
assessment to the second assessment revealed 
no statistically significant interaction for 
ability status and condition, F (1j71) = .573, 
p — .452. However, a main effect for 
condition (graphing calculator use on 
Assessment 1 vs. graphing calculator use on 
Assessment 2) was found, F (l j Vj = 26.118, 
p < .000, t] p 2 = .269; /? = .999. This suggests 
that students who had access to a graphing 
calculator on the second but not the first 
assessment showed greater gains (from 


Assessment 1 to Assessment 2) than students 
who had access to a graphing calculator on 
the first assessment but not the second. A 
main effect for students’ gain scores was not 
found for ability status (students with 
disabilities vs. students without disabilities), 
F( 155 ) = .904 ,p = .345, suggesting that students 
with disabilities did not differ from students 
without disabilities on their change scores 
from the first assessment to the second 
assessment. 

Figure 2 depicts the graphical representation 
of the data of change scores for students with 
and without disabilities by condition. The 
graph indicates illustrates the change in scores 
from Assessment 1 to Assessment 2 for the 
two groups of students (students with and 
without disabilities) via the two conditions 
(calculator and then no calculator and no 
calculator and then calculator). It indicates 
that students who had a calculator on 
Assessment 2 had positive change scores - 
they did better on Assessment 2 than 
Assessment 1, regardless of ability (although 
students with a disability were slightly higher), 
whereas students who had a calculator on 
Assessment 1 had a negative change score, 
meaning they did better on Assessment 1 than 
Assessment 2. 


Assistive Technology Outcomes and Benefits / 23 


Fall 2008, Vol.5, Num. 1 


Change Scores from Assessment 1 to Assessment 2 



Figure 2\ Change score from first assessment to second assessment across condition and ability. 


All students, regardless of ability, answered 
more problems correctly on the mathematics 
assessment when they had access to a 
graphing calculator. For those who had access 
to a graphing calculator on Assessment 1, 
students with disabilities answered an average 
of 3.17 questions correctly and students 
without disabilities answered an average of 8 
questions correctly (see 1 for means). This is 
in contrast to students who did not have 
access to a graphing calculator on the first 
assessment, in which students with disabilities 
averaged 2.31 correct responses and students 
without disabilities averaged 5.93 correct 
responses. Similarly on Assessment 2, 
students with disabilities who had access to a 


graphing calculator answered an average of 
5.23 questions correctly and students without 
disabilities answered an average of 8.63 
correcdy, as opposed to students who did not 
have access to a graphing calculator (average 
of 1.17 correct for students with disabilities 
and 4.07 for students without disabilities). The 
change score in Condition 1 for students with 
a disability was a -2.0 and -3.93 for students 
without disabilities. However, in Condition 2 
the change score for students with disabilities 
was +2.92 as compared to +2.7 for students 
without disabilities (refer to Table 1). 

The /-tests for each condition with ability 
status (students with disabilities and students 


Table 1 

Means for Scores by Ability Status and Condition 


Condition 1 Condition 2 


SWD (6) 

SWOD (29) 

SWD 1131 

SWOD (27) 


Assessment 1 

3.17 

8 

2.31 

5.93 

Assessment 2 

1.17 

4.07 

5.23 

8.63 

Change score 

-2.0 

-3.93 

+2.92 

+2.7 


Note: SWD refers to students with disabilities; SWOD refers to students without disabilities 
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without disabilities) as a factor suggest that 
graphing calculators are not a valid 
accommodation. The independent /-test for 
Condition 1, with the dependent variable of 
scores on Assessment 1 and ability as a factor, 
was significant, / (33) = 2.453, p = .02, favoring 
students without disabilities. The independent 
t-test for Condition 2, with the dependent 
variable of scores on Assessment 2 and ability 
as a factor, was significant, / (38) = 2.508, 
p — .017, also favoring students without 
disabilities. The significant /-tests suggest that 
students with disabilities did not benefit more 
than students without disabilities when given 
access to a graphing calculator as an 
accommodation; in fact, students without 
disabilities benefited more. 

Discussion 

This study sought to answer the question: 
Does the use of a graphing calculator result in 
performance differences on standards-based, 
open-ended, problem-solving assessments for 
students with and without disabilities? The 
results indicate that both students with and 
without disabilities answered more open- 
ended, problem-solving questions correctly 
with access to a graphing calculator than 
without. However, the results further suggest 
that graphing calculators are not a valid 
assessment accommodation, given the 
definition of a valid accommodation, as one in 
which students with disabilities benefit to a 
greater extent than students without 
disabilities (Elbaum, 2007). 

The findings of this study indicate that both 
students with and without disabilities 
performed better on standards-based, open- 
ended, problem-solving mathematics 
assessments when they had access to a 
graphing calculator. This is not necessarily 
surprising given that a calculator can reduce 
students’ mental math mistakes. These 
findings both support and extend previous 
research regarding calculator use on 


mathematics assessments and students with 
disabilities (Bouck, in press; Bouck & Bouck, 
2008; Fuchs et al., 2000; Shaftel et al., 2003). 
The results support previous research by 
replicating that access to a calculator can 
result in performance gains by students with 
and without disabilities; yet do not support 
calculators as a valid accommodation on 
mathematics assessments. The lack of 
students with disabilities benefiting more 
from a calculator might suggest that, while 
calculators help minimize the mental math 
mistakes of these students, they do not 
compensate for lower conceptual 
understanding. Lower conceptual 

understanding by some students with 
disabilities as compared to some students 
without disabilities might explain the 
statistically significant benefit to students 
without a disability when given a calculator. 
However, this interpretation from this limited 
research is not intended to be used as a 
rationale for denial of services to students 
with disabilities or a dismissal of calculator use 
by either population. 

Outcomes and benefits 

Students with disabilities have historically 
performed worse in mathematics than 
students without disabilities. For example, 
students with disabilities often stmggle with 
automaticity of basic facts, computation 
problems, and problem-solving (Cawley, 
Parmar, Fley, Salmon, & Roy, 2001; Jitendra, 
DiPipi, & Perron-Jones, 2002; Montague, 
1992; Woodward & Montague, 2002). The 
data from this study suggest that calculators as 
an assistive technology tool cannot solve all 
the mathematical challenges faced by students 
with disabilities. A lack of conceptual 
understanding of a mathematical idea cannot 
be overcome through the use of a calculator. 
While calculators can reduce mental mistakes 
or students’ struggle with basic facts, which is 
a positive result, they cannot generate an 
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understanding of a mathematical concept if a 
student does not possess it. 

Hence, a need exists to increase the 
mathematical conceptual understanding of 
students with disabilities. Additional 
instruction focused on making sense and 
understanding mathematical ideas rather than 
efficiency with procedures is needed for 
students with disabilities. This is not to say 
that students with disabilities should not be 
given access to a calculator, as clearly these 
students benefited from having access (i.e., 
answered more correctly with a calculator 
than without). Allowing students with 
disabilities access to a calculator has the 
potential to give teachers greater insight into 
students’ tme mathematical knowledge bases 
when they are not hung-up by mental math or 
basic facts mistakes. 

In conclusion, the data from this study on 
calculators as an assistive technology 
accommodation on mathematical assessments 
suggested all students, regardless of ability 
status, performed better on the open-ended, 
problem-solving assessments aligned to state 
standards when they had access to a graphing 
calculator. Yet, the data also suggested that 
graphing calculators are not a valid 
accommodation when using Elbaum’s (2007) 
definition of a valid assessments 
accommodation, as students with disabilities 
did not benefit to a greater extent than 
students without disabilities from access to 
this tool. This is of particular importance 
given that 14 states within the United States 
of America allow calculators as 
accommodations on large-scale assessments, 
14 allow them under certain circumstances, 1 
allows them but with implications for scoring, 
7 allow them under specific circumstances and 
with implications for scoring, and 5 consider 
them a non-standard accommodation but 
with no implications for scoring (Lazarus, 
Thurlow, Lail, Eisenbraun, & Kato, 2006). 
Educators and policymakers need to consider 


the research when deciding if and/or when 
calculators are a valid accommodation and 
should be allowed on assessments (Fuchs et 
al., 2005). 

L imitations 

This study has a few limitations in that only 
one school district was involved. It was 
conducted with a limited number of students 
in total and specifically students with 
disabilities. Another limitation involved 
missing data, which was a result of the length 
of the assessments. Twenty-eight open- 
response, problem-solving questions were too 
many for students with and without 
disabilities to complete in one class period. 
Students who did not finish either assessment 
employed different test-taking strategies, such 
as starting at the beginning and finishing as 
much as one could or skipping around and 
answering questions the student thought s/he 
knew. Finally, data was not analyzed at the 
level of type of disability, rather disability 
classifications were aggregated together. Data 
also was not aggregated for students with 
disabilities who were indicated to need a 
calculator as an accommodation versus those 
students whose IEP did not specify as such. 
While accommodations are meant to be 
determined on an individual student level 
given a student’s strengths and challenges, this 
study sought to begin to examine calculators 
as assessment accommodations. Future 
research should address the limitations of this 
study. 

Future Directions 

Additional research is needed regarding 
mathematics assessment accommodations, 
particularly for standardized tests following 
mandates under NCLB. Specifically, 
additional research is needed to examine 
graphing calculators as valid 

accommodations-both in the classroom for 
daily use and on assessments. Future research 


Assistive Technology Outcomes and benefits / 26 



Fall 2008, Vol.5, Num. 1 


should replicate studies like this as well as 
extend the ages examined (i.e., elementary and 
high school). Finally, research should explore 
calculators as accommodations on a range of 
assessment types, such as computation 
problem, problem-solving questions, and in 
situations simulating standardized testing 
situations as well as other mathematics strands 
(i.e., geometry). 
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